Papers by Zheng Xin Yong
Preference Tuning For Toxicity Mitigation Generalizes Across Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Detoxifying multilingual Large Language Models (LLMs) has become crucial due to their increasing global use. |
| Approach: | They propose to use English preference tuning to study cross-lingual detoxification of LLMs. |
| Outcome: | The proposed method reduces toxicity in multilingual LLMs by reducing the probability of mGPT-1.3B generating toxic continuations across 17 languages. |
The State of Multilingual LLM Safety Research: From Measuring The Language Gap To Mitigating It (2025.emnlp-main)
Copied to clipboard
| Challenge: | a systematic review of 300 publications reveals a language gap in LLM safety research . even high-resource non-English languages receive little attention, authors note . |
| Approach: | They propose to focus on safety evaluation, training data generation, and crosslingual safety generalization based on their findings. |
| Outcome: | The authors suggest that the field can develop more robust, inclusive safety practices for diverse global populations. |
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts (2022.acl-demo)
Copied to clipboard
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, Alexander Rush
| Challenge: | PromptSource is a system for creating, sharing, and using natural language prompts . prompts are used to train and query language models in zero-shot learning settings . |
| Approach: | PromptSource is a system for creating, sharing, and using natural language prompts . et al.: using prompts to train and query language models is emerging area in NLP . they propose a templating language for defining data-linked prompts, a user interface that iterates on prompt development . |
| Outcome: | PromptSource is a system for creating, sharing, and using natural language prompts . it has a templating language for defining data-linked prompts and a community-driven set of guidelines . |
What Language Model to Train if You Have One Million GPU Hours? (2022.findings-emnlp)
Copied to clipboard
Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
| Challenge: | Recent years have seen the advent of large language models characterized by emergent capabilities arising from sheer scale alone. |
| Approach: | They propose to use a multilingual model to compare performance to the English-only model by ablation at the billion-parameter scale. |
| Outcome: | The proposed model is based on a multilingual model and its performance against the English-only model. |
Semi-supervised Deep Embedded Clustering with Anomaly Detection for Semantic Frame Induction (2020.lrec-1)
Copied to clipboard
| Challenge: | Empirical results show that definitions provide contextual information for representing and characterizing the frame membership of lexical units. |
| Approach: | They propose a two-step frame induction process to remove lexical units that cannot fit into existing frames in Berkeley FrameNet. |
| Outcome: | The proposed method outperforms state-of-the-art methods in both steps of the frame induction process. |
Crosslingual Generalization through Multitask Finetuning (2023.acl-long)
Copied to clipboard
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel
| Challenge: | Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. |
| Approach: | They apply multitask prompted finetuning to pretrained multilingual models and generate variants called BLOOMZ and mT0. |
| Outcome: | The proposed models can generalize to non-English languages that have never been seen before. |
Frame Shift Prediction (2022.lrec-1)
Copied to clipboard
| Challenge: | Frame shift is a cross-linguistic phenomenon in translation which results in corresponding pairs of linguistic material evoking different frames. |
| Approach: | They propose a task to predict cross-linguistic frame-to-frame correspondence and propose auxiliary training to learn cross-lingual frame-by-frame correlation. |
| Outcome: | The proposed task can learn cross-linguistic frame-to-frame correspondence and predict frame shifts in a Berkeley FrameNet-like configuration. |
LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word-to-word translations from labeled task data in low-resource languages have limited lexical overlap with task data. |
| Approach: | They propose a method that generates low-resource-language classification task data at scale using bilingual lexicons. |
| Outcome: | The proposed method improves on 17 low-resource languages with bilingual lexicons compared with existing models on sentiment analysis and topic classification tasks. |
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting (2023.acl-long)
Copied to clipboard
Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, Alham Fikri Aji, David Ifeoluwa Adelani, Khalid Almubarak, M Saiful Bari, Lintang Sutawika, Jungo Kasai, Ahmed Baruwa, Genta Winata, Stella Biderman, Edward Raff, Dragomir Radev, Vassilina Nikoulina
| Challenge: | Existing language adaptation strategies for multilingual models are limited to 46 languages . a new language is added to the model to improve zero-shot prompting performance . |
| Approach: | They apply existing language adaptation strategies to BLOOM and benchmark its zero-shot prompting performance on eight new languages in a resource-constrained setting. |
| Outcome: | The proposed model can be extended to other languages without incurring prohibitively large costs. |
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have sparked concerns about their safety. |
| Approach: | They propose a method to identify safety-related information in the model parameter space . they propose to use a few adversarially chosen examples to fine-tune LLMs . |
| Outcome: | The proposed method can break safety alignment in multilingual LLMs using a few examples . it also shows that the proposed method jailbreaks LLM models adapted to new languages . |
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (2024.emnlp-main)
Copied to clipboard
Holy Lovenia, Rahmad Mahendra, Salsabil Akbar, Lester James Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno Kampman, Joel Moniz, Muhammad Habibi, Frederikus Hudi, Jann Montalan, Ryan Hadiwijaya, Joanito Lopo, William Nixon, Börje Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Irawan, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Ryanda, Sonny Hermawan, Dan Velasco, Muhammad Kautsar, Willy Hendria, Yasmin Moslem, Noah Flynn, Muhammad Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Tai Chia, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Aji, Sedrick Keh, Genta Winata, Ruochen Zhang, Fajri Koto, Zheng Xin Yong, Samuel Cahyawijaya
| Challenge: | Southeast Asia (SEA) is home to over 1,300 indigenous languages and 671 million people . prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA . |
| Approach: | They propose to provide a resource center that provides standardized corpora in nearly 1,000 SEA languages across three modalities. |
| Outcome: | a new benchmark assesses the quality of AI models on 36 SEA languages across 13 tasks . the results highlight the importance of SEA as a culturally diverse region . |
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges (2023.findings-acl)
Copied to clipboard
| Challenge: | Code-Switching is a common phenomenon in written text and conversation . it is not so common to observe code-switching in spoken language and not in written language . |
| Approach: | They present a systematic survey on code-switching research in natural language processing to understand the progress of the past decades and conceptualize the challenges and tasks on the topic. |
| Outcome: | The proposed model combines linguistic theories and machine learning techniques to understand the code-switching phenomenon. |
Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingualism is widespread around the world and code-switching (CSW) is a common practice among different language pairs/tuples across locations and regions. |
| Approach: | They propose to use existing CSW data sets to improve the representativeness of CSW datasets. |
| Outcome: | The proposed model lacks representativeness due to location-based, socio-demographic and register variation in CSW data. |